Embodied sentence comprehension 1 EMBODIED SENTENCE COMPREHENSION

ions are being discussed in other chapters in this book (Barsalou & WiemerHastings, this volume; Gibbs, this volume; Prinz, this volume). 6.0 Conclusion and outlook We started this chapter by comparing two perspectives on cognition in general and on language comprehension in particular. According to the traditional perspective, language comprehension involves the activation and integration of discrete and abstract building blocks of meaning. These building blocks are most commonly represented as propositions or equivalently in semantic networks (in which a pair of nodes are the arguments and the link between them the predicate). We have demonstrated that this general view cannot account for a range of recent findings about language comprehension obtained in our experiments and in experiments by others. According to the alternative view, cognition in general and language comprehension in particular involve the activation and integration of experiential traces in the construal of a situation. These traces are activated by linguistic constructs, which are experiential representations in their own right. Language can be viewed as a sequence of cues modulating the comprehender’s attention to a referential world, which is simulated by integrating experiential traces. As we have shown, this view can account for the recent findings we have discussed in this chapter. Moreover, it generates a volley of testable predictions regarding language comprehe nsion, for example about the role of perspective in language comprehension. Embodied sentence comprehension 30 However, it is clear that the view we have attempted to outline here is in need of further articulation. The challenge for researchers adopting an experiential perspective is to further articulate the theoretical frameworks, keeping them consistent with what is known about the brain, and test them in elegant and convincing experiments. We are optimistic that these goals are within our reach. Our focus has been on the role of visual representations in language comprehension. This is because our empirical research thus far has focused on this phenomenon, primarily because it was motivated in part by the goal to show the limitations of amodal propositional representations. However, it is clear that embodied language comprehension involves more than just visual representations. For example, there is behavioral evidence that language comprehension may involve the activation of motor programs (Glenberg & Kaschak, 2002). The sympathetic activation of neurons in the premotor cortex, first observed in monkeys, is thought to underlie action understanding and therefore to mediate language comprehension (Rizzolatti & Arbib, 1998). MacWhinney (this volume) provides an extension of these ideas. A full–fledged theory of embodied language comprehension should include both perceptual and action simulations. Moreover, it should be capable of explaining their relations (do they occur simultaneously, are they integrated into a coherent representation or are they independent, how is their construal cued by linguistic constructs). The claims made by proponents of embodied comprehension, for example about the activation of visual representations during language comprehension, may at the same time seem trivial and counterintuitive. They will seem trivial to the lay person, or even to people with great expertise in the use of language, such as novelists and poets. Of course, Embodied sentence comprehension 31 words can be used to conjure up images in the reader’s mind! However, these same claims will seem counterintuitive to researchers trained in traditional cognitive science. To them, the claim that meaning can be captured by experiential representations does not make sense. For one, the claim opens the door to the homunculus problem, and thus to an infinite regress. If there are pictures in the head, then there must be a little person in there looking at the pictures. And if so, who’s in that person’s mind? There are two responses to this criticism. First, this problem also seems to apply to the amodal view. After all, where is the little person reading all those quasi-linguistic propositions entering and leaving the revolving door of working memory? Second, and more importantly, the claim is not that there are pictures in the mind. Rather, the claim is that traces of visual and other experiences are (partly) reactivated and recombined in novel ways by associated words (Barsalou, 1999). In this sense, language comprehension is the vicarious experiencing of events. Speakers and writers carefully orchestrate linguistic constructs so that experiential traces in their audience’s minds can be recombined in novel ways to produce novel experiences. Embodied sentence comprehension 32